Apparatus and method for processing 3D video data

ABSTRACT

A method comprising: obtaining a disparity map for one of a left eye image and a right eye image, the disparity map indicating the horizontal displacement between corresponding pixels in the left eye image and the right eye image; generating a luminance representation of the disparity map, wherein a disparity unit which defines the resolution of the disparity map, is mapped to a corresponding luminance value; and companding the luminance value for transmission over a luminance channel is described.

BACKGROUND

1. Field of the Disclosure

The present disclosure relates, in general, to an apparatus and methodfor transmitting disparity maps.

2. Description of the Related Art

3D television and film is becoming increasingly popular. Typically, whengenerating 3D content, a disparity map is formed which identifies theamount of horizontal separation between the left and right eye versionsof each image. Specifically, a map is formed showing the amount ofdifference between the left and right eye versions of each pixel in eachimage. The disparity map may be used for many reasons. One example is todetermine where to place graphics within the 3D image. This ensures thata graphic is not placed in an inappropriate location on the screen, andmore specifically, so that the graphic does not occlude part of or allof the 3D image that is in front of the graphic in 3D space. Thedisparity map may also be used to determine where to locate subtitles,such as closed captioning.

It is known to place any captions or graphics in front of all otherobjects within the 3D image. In order to achieve this, the caption isplaced within the 3D video using a vision mixer. Placing the captions orgraphics in this location allows the graphic not to be occluding part orall of the 3D image and is particularly useful in real-time insertion ofa caption as a maximum amount of disparity for each pixel in the imagecan be determined and never exceeded. However, positioning the graphicin such a way has the disadvantage of causing discomfort to the viewerover an extended period because the viewer must look at the caption andthen to re-adjust their eyes to view the 3D image.

It would be desirable to find a way of determining the disparity mapquickly so that the caption can be placed at an appropriate place withinthe 3D environment.

Moreover, as mechanisms currently exist to allow the inclusion ofcaptions into a 3D image, it would be desirable to be backwardlycompatible with current technology. In particular, it would be useful toenable where to place the caption using current vision mixers.

SUMMARY OF THE DISCLOSURE

According to an aspect of the disclosure, there is provided a methodcomprising: obtaining a disparity map for one of a left eye image and aright eye image, the disparity map indicating the horizontaldisplacement between corresponding pixels in the left eye image and theright eye image; generating a luminance representation of the disparitymap, wherein a disparity unit which defines the resolution of thedisparity map, is mapped to a corresponding luminance value; andcompanding the luminance value for transmission over a luminancechannel.

This allows a disparity map to be transferred over the luminance channelof, for example, an HD-SDI connection. This allows the disparity map tobe used by a conventional vision mixer.

The luminance may be companded using a logarithmic function. This isuseful because a logarithmic function has a steep roll-off allowing abetter resolution in commonly used disparities.

The luminance may companded using a segmented linear function.

The companding function may selected in dependence on the size of thedisplay onto which the left eye image and the right eye image are to bestereoscopically displayed so that the gradient of the compandingfunction is greatest at a disparity suited to the display.

This allows a better resolution in commonly used disparities for thedisplay

In the case of a logarithmic function being used, the logarithmicfunction may be

${F(x)} = {512 + {{round}\left( {512\mspace{14mu}{{sign}(x)}\frac{\ln\left( {1 + {\mu{\frac{x}{Dispscale}}}} \right)}{\ln\left( {1 + \mu} \right)}} \right)}}$

-   -   Between the limits—Dispscale≦x≦+Dispscale    -   where Dispscale is the number of disparity units; μ is a        variable that controls the magnitude of the logarithmic        function; sign(x) means±(x) and the round rounds the function to        the nearest integer value.

In the case of a linear segmented function being used, wherein thelinear function may be segmented into a first and second segment,wherein the companding function of the first segment is

${F(x)} = {{512 + {{round}\left( \frac{x}{8} \right)} - 2048} < x < 2048}$

-   -   And the companding function for the second segment is

$\begin{matrix}{{F(x)} = {512 + {{round}\left( {\frac{x}{256} + {248\mspace{14mu}{{sign}(x)}}} \right)}}} & {{- {DispScale}} \leq x \leq {- 2048}} \\\; & {2048 \leq x \leq {DispScale}}\end{matrix}$

Where Dispscale is the number of disparity units; sign(x) means±(x) andthe function round rounds the function to the nearest integer value.

The method may further comprise generating an error signal, the errorsignal being the difference between the companded luminance value andthe luminance value; and converting the error signal for transmissionover a chrominance channel.

In this case, the method may further comprise expanding the compandedluminance value; retrieving the error signal from the chrominancechannel and adding the retrieved error signal to the expanded luminancevalue to form the original luminance value.

The method may further comprise retrieving the companded luminance valuefrom a look-up table where the companded luminance value is stored inassociation with the luminance value.

In this case, the method may further comprise retrieving the errorsignal from a look up table where the error signal is stored inassociation with the luminance value.

According to an aspect of the disclosure, a vision mixer comprising: aluminance channel and a chrominance channel configured in use to receivea companded luminance signal over the luminance channel and an errorsignal over the chrominance channel; and a video mixing deviceconfigured in use to receive a caption to be inserted into a video, andto provide the location and depth at which the caption is to be insertedinto the video, wherein the location is determined in accordance withthe received companded luminance signal.

The vision mixer may further comprise: an expander configured in use toconvert the received companded luminance signal into a second luminancesignal; and an adder configured in use to add the second luminancesignal to the received error signal.

According to another aspect, there is provided an encoder comprising: adisparity map obtaining device configured in use to obtain a disparitymap for one of a left eye image and a right eye image, the disparity mapindicating the horizontal displacement between corresponding pixels inthe left eye image and the right eye image; a luminance value generatorconfigured in use to generate a luminance representation of thedisparity map wherein a disparity unit, which defines the resolution ofthe disparity map, is mapped to a corresponding luminance value; and acompanding device configured in user to compand the luminance value fortransmission over a luminance channel.

The luminance may be companded using a logarithmic function.

The luminance may be companded using a segmented linear function.

The companding function may be selected in dependence on the size of thedisplay onto which the left eye image and the right eye image are to bestereoscopically displayed so that the gradient of the compandingfunction is greatest at a disparity suited to the display.

In the case of a logarithmic function being used, the logarithmicfunction may be

${F(x)} = {512 + {{round}\left( {512\mspace{14mu}{{sign}(x)}\frac{\ln\left( {1 + {\mu{\frac{x}{Dispscale}}}} \right)}{\ln\left( {1 + \mu} \right)}} \right)}}$

-   -   Between the limits—Dispscale≦x≦+Dispscale    -   where Dispscale is the number of disparity units; μ is a        variable that controls the magnitude of the logarithmic        function; sign(x) means±(x) and the round rounds the function to        the nearest integer value.

In the case of a segmented linear function being used, the linearfunction may be segmented into a first and second segment, wherein thecompanding function of the first segment is

${F(x)} = {{512 + {{round}\left( \frac{x}{8} \right)} - 2048} < x < 2048}$

-   -   And the companding function for the second segment is

$\begin{matrix}{{F(x)} = {512 + {{round}\left( {\frac{x}{256} + {248\mspace{14mu}{{sign}(x)}}} \right)}}} & {{- {DispScale}} \leq x \leq {- 2048}} \\\; & {2048 \leq x \leq {DispScale}}\end{matrix}$

Where Dispscale is the number of disparity units; sign(x) means±(x) andthe function round rounds the function to the nearest integer value.

The encoder may further comprise an error signal generator configured inuse to generate an error signal, the error signal being the differencebetween the companded luminance value and the luminance value; and anerror signal converting device configured in use to convert the errorsignal for transmission over a chrominance channel.

The encoder may further comprise a look up table, wherein the look uptable is configured in use to store the companded luminance value inassociation with the luminance value, and the companding device isconfigured in use to retrieve the companded luminance value from thelook-up table.

In this case, the look-up table may be configured in use to store theerror signal, where the error signal is stored in association with theluminance value and the companding device is further configured toretrieve the error signal from the look-up table.

According to another aspect, there is provided a decoder comprising: aluminance channel input and a chrominance channel input configured inuse to receive a companded luminance signal via the luminance channelinput and an error signal via the chrominance channel input; and anexpander configured in use to convert the received companded luminancesignal into a second luminance signal.

The decoder may further comprise an adder configured in use to add thesecond luminance signal to the received error signal.

According to another aspect, there is provided computer softwarecontaining computer readable instructions which, when loaded onto acomputer, configure the computer to perform a method according to anyone of the embodiments.

According to another aspect, there is provided a computer programproduct configured to store the computer software according toembodiments therein or thereon.

BRIEF DESCRIPTION OF DRAWINGS

The above and other objects, features and advantages of the disclosurewill be apparent from the following detailed description of illustrativeembodiments which is to be read in connection with the accompanyingdrawings, in which:

FIG. 1 shows a system according to embodiments of the presentdisclosure;

FIG. 2 shows a diagram explaining the use of displaying a disparity mapon a system according to embodiments of the present disclosure;

FIG. 3A shows an encoder according to an embodiment of the presentdisclosure;

FIG. 3B shows a HD-SDI convertor according to embodiments of the presentdisclosure;

FIG. 4 shows a vision mixer according to embodiments of the presentdisclosure; and

FIG. 5 shows a flow chart explaining embodiments of the presentdisclosure.

DESCRIPTION OF THE EMBODIMENTS

Referring to FIG. 1, a system 100 for capturing a 3D image and fordetermining a disparity map is shown. A 3D camera 105 captures an image.As would be appreciated, the 3D camera 105 captures an image for theleft eye of a viewer and an image for the right eye of the viewer. Theleft and right eye images are horizontally displaced from one another.The left eye image and the right eye image are sent to an encoder 110.The 3D camera 105 and the encoder 110 may be connected using a wired orwireless connection.

Additionally connected to the encoder 110 is a user interface 125. Theuser interface 125 may be a keyboard or a mouse or indeed may be tablettype device such as the Sony® tablet S or a smartphone such as the Sony®Xperia S. The user interface 125 may be connected to the encoder 110using a wireless or wired connection. A storage medium 130 is alsoconnected to the encoder 110. The storage medium 130 may be an opticalor magnetically readable medium or may be a solid state storage medium.The storage medium 130 is used to store the captured left eye and righteye images. These images can then be accessed by an editing suite wherecaptions may be inserted. For brevity, an explanation of the editingsuite is not included as it does not clarify embodiments of thedisclosure.

Also connected to the encoder 110 is a vision mixer 120. The visionmixer 120 is connected to the encoder 110 using a connection. Althoughmany types of connection are envisaged, such as an SDTI (Serial DataTransport Interface) connection in embodiments an HD-SDI (HighDefinition—Serial Digital Interface) connection is used. As will beexplained later, the disparity map information produced by the encoder110 is digital information that indicates the horizontal distancebetween pixels in the right eye image and the left eye image. Typically,a disparity map is generated for each of the left and right eye image.So, the disparity map for the right eye image indicates the horizontaldisplacement of the same pixel in the left eye image.

As the disparity map is not video data, it may be transferred over theSDTI connection in its original form. In embodiments there is a transferof the disparity map to the vision mixer 120 as luminance data over theHD-SDI connection. By transferring the disparity map data to the visionmixer 120 over the luminance data channel of the HD-SDI connectionallows the caption to be positioned with the appropriate disparity sothat it may be placed at the appropriate distance from the display whenviewed by a user. This not only allows the caption to be located at theappropriate depth to avoid occluding the objects in the 3D content, butproviding the luminance data in HD-SDI also assists with backwardscompatibility with current vision mixers.

As will be appreciated by the skilled person, although the followingexplains using the disparity map with the vision mixer 120, thedisclosure is not so limited. As will be appreciated, the disparity mapmay be fed to a display instead of or in addition to the vision mixer120 in FIG. 1. The disparity map may be used in a display to provideimproved visual representation of the disparity map. In other words, thedisparity map may be provided by an external source and may bepre-calculated.

As the HD-SDI connection transfers video information, and in particular,chrominance and luminance information only, the disparity mapinformation must be processed by the encoder 110 before being sent overthe connection to the vision mixer 120. In particular, the disparity mapdata must be converted into luminance data so that for a given value ofdisparity, there is a given value of luminance.

After conversion into a luminance value, the disparity map informationmay be used by a conventional vision mixer without modification of thevision mixer. This is because HD-SDI is a typical format used inbroadcast to view images. Alternatively, as will be explained, in thecase of using the disparity map information on a conventional visionmixer, the disparity map information will likely include errors.However, in embodiments of the present disclosure, a vision mixer 120 isprovided in which the errors are corrected.

In order to illustrate a disparity map, a displayed disparity map isshown. In a left eye image 205, a circle and square are to be shown in3D. The right eye image 210 is shown below the left eye image. As can beseen, the horizontal distance between the square in the left eye imageand the right eye image is greater than the horizontal distance betweenthe circle in the left eye image and the right eye image. Accordingly,if the left eye image and the right eye image were displayedstereoscopically, the circle would appear closer to the viewer 230 thanthe square. In other words, the distance d which indicates the perceiveddistance of the square from the image plane would be less than d′ whichindicates the perceived distance of the circle from the image plane.

If, for each value of disparity, a corresponding value of luminance wasgenerated, a disparity map 235 for the left eye image is created. Thevisualisation of this disparity map is shown in 235. Similarly thevisualisation disparity map 240 for the right eye image is shown. As isevident from the figures, the disparity map of the square is blackwhereas the circle is grey (indicated by the hashed lines). In otherwords, if the disparity map is displayed using the luminance channel,the greater the disparity at a pixel position in the left or right eyeimage, the lighter the pixel in the displayed disparity map. So, thismeans that for each value of disparity in the disparity map, there is acorresponding value of luminance. This illustrates that for any givenvalue of disparity, a luminance value exists. Therefore, the visionmixer 120 may use luminance as a key to determine where to locate thegraphic.

Uses of Disparity Map Over Luminance Channel

As noted in FIG. 2, as one particular value of luminance is mapped toone value of disparity, it is possible to the use the luminance value asa key for graphics insertion.

Keying is used in vision mixers to determine where to insert a captionor graphic. It is known to use chroma keying when deciding where tolocate a graphic. For example, so-called “Green Screen” is known wherean actor or presenter stands in front of a green screen. The visionmixer knows that it must insert the graphic wherever the chroma signalincludes that particular shade of green.

In embodiments of the present disclosure, luminance keying is used todetermine where the caption should be located in the video and theamount of disparity to provide to a caption. In the case that a captionhaving a disparity map (converted into luminance data) and video contenthaving a second disparity map (converted into luminance data) are fedinto the vision mixer, by keying a particular luminance value, thevision mixer knows where the graphic or caption should be located in thevideo so to ensure that the caption or graphic does not occlude theobject in the 3D image. As will be explained, this is achieved by thevision mixer keying to a particular luminance value, and if the captionwould occlude the content, a minimum luminance value being taken. Thevision mixer 120 then provides information so that the caption can beinserted to the content at a disparity provided by this minimumluminance value.

Referring to FIG. 3A, the left and right image from the camera 105 isfed into a controller 310. The controller 310 is, in embodiments, amicroprocessor. The controller 310 operates using a computer program.The computer program is stored on controller storage medium 320. Thecontroller storage medium 320 may be a magnetically readable, opticallyreadable or solid state storage medium. Indeed the computer program maybe embodied as a signal which may be transferred over the Internet orthe like. The controller storage medium 320 is connected to thecontroller 310.

Additionally connected to the controller 310 are a user interfacecontroller 305 and a HD-SDI convertor 315. The HD-SDI convertor 315 willbe described in more detail with reference to FIG. 3B. The userinterface controller 305 is connected to the user interface 125. Whenthe user controls the user interface 125, the user interface controller305 generates controls signals which are input into the controller 310.

The controller 310 generates the disparity map for each of the left andright images captured by the 3D camera 105. The mechanism by which thedisparity map is generated is known. For example, the disparity map maybe generated using image based block matching techniques or depth sensortechniques such as laser or infra-red rangers or the like. However, asnoted above, the disparity map may be provided externally and may not becreated within the controller 310.

The generated disparity map for each the left eye and right eye image isfed to the HD-SDI convertor 315. The HD-SDI convertor 315 converts thedisparity map into a luminance signal and optionally an error signal.The luminance signal is fed to the vision mixer over the luminancechannel and the error signal is fed to the vision mixer 120 over thechrominance channel. The purpose of the error signal will becomeapparent hereinafter.

An unconverted disparity map is also fed to the storage medium 130 forstorage thereon.

Conversion of the Disparity Map into a Luminance and optional ErrorSignal

In order to convert the disparity map into a luminance signal and anoptional error signal, the resolution of the disparity map isdetermined. The resolution of the disparity map means the minimum amountof distance between a pixel in one image and the corresponding pixel inthe other image that can be observed.

In some situations, it is desirable to provide a maximum disparity of ±1screen width. This may be desirable in situations where there is a verysmall screen size used to view the 3D image and so the distance betweenpixels is very small. An example of this would be a 3D enabledsmartphone that has a screen made of lenticular lenses.

However, in most situations the amount of disparity required between theleft eye image and the right eye image is much smaller as a percentageof screen width. For example for video captured for display on atelevision, the disparity is typically +3% to −2% of screen width andfor video captured for display in a movie theatre, the disparity istypically +1% to −2% of screen width. However, it is understood thatother levels of disparity may be typical. The other levels of disparitymay depend, for example, on screen size and/or the video effectrequired.

In some situations, it is desirable to provide a minimum disparity of ±1/32 of a pixel. This sub-pixel disparity allows for most 3D videoeffects, and is especially suited for 3D images to be displayed in amovie theatre. Again, the skilled person would appreciate though thatany level of minimum disparity may be selected.

Therefore, in the above case where a minimum disparity of ± 1/32 of apixel is supported, the minimum resolution (termed “disparity units”hereinafter) is 1/32 pixel.

In a full High Definition display (which we mean to be a 1080i/p displayhaving a horizontal width of 1920 pixels), the number of disparity unitswhich must be supported by the conversion is 32*1920=±61440 disparityunits. In other words, and more generally, for a maximum disparity of ±1frame width, the number of disparity units that would, in embodiments,be supported by the HD-SDI convertor 315 is given by equation (1)Number of disparity units=±number of pixels in horizontal direction ofscreen*number of sub-pixels to make one pixel  [Equation (1)]

In order to provide the dynamic range of ±61440 disparity units when fedto the vision mixer 120 on the luminance channel, the luminance value ofeach pixel in the disparity map must be represented by a 17 bit number.Clearly, the number of bits that are required to represent the number ofthe disparity units is dependent upon the both the minimum resolution ofthe disparity unit and the size of the screen upon which the video willbe displayed. For a given number of sub-pixels (i.e. 32 disparity unitsper pixel in the above case), for a larger screen size, a larger dynamicrange is required. This may necessitate a larger number of bits torepresent the disparity units. However, it may be desirable to maintainthe size of dynamic range and alter the minimum resolution for any sizeof screen upon which the video is to be provided. So, for example,maintaining the dynamic range of ±61440 disparity units, if the screensize is 2048 pixels wide, the minimum resolution of the disparity map is1/30^(th) of a pixel.

As already noted, it is desirable to view the disparity map (with adynamic range represented by 17 bits) as a luminance signal over anHD-SDI connection. However, as is understood by the skilled person, theluminance channel of the HD-SDI connection supports less than 17 bitsdynamic range. Typically, the luminance channel of the HD-SDI supports a10 or 12 bit number. Similarly, the chrominance channel of the HD-SDIconnection supports less than 17 bits dynamic range, and typically thisis a 10 bit or 12 bit number. For convenience, the following explanationrelates to the 10 bit luminance and chrominance channel.

In embodiments, the 17 bit number which represents the pixel value of apixel in the disparity map is converted into a 10 bit number.

Conversion of the 17 Bit Disparity Unit Number Into 10 Bit HD-SDILuminance Value

In order to convert the 17 bit disparity unit number into a 10 bitHD-SDI luminance value, in embodiments, the video image is companded.Companding is a term of art and allows signals with a large dynamicrange to be transmitted over a channel having a smaller dynamic rangecapability. The process of companding, however, is a lossy process.

In order to utilise the 10 bit luminance channel, a logarithmic encodingfunction is used. This is because typically the amount of disparitybetween the left and right image depends to an extent on the size of thescreen upon which the 3D content will be viewed. For example, as notedabove, when the content is to be viewed on a television ±3% to −2% ofscreen width is common for upper limits on disparity.

Therefore, by selecting a logarithmic encoding function, which has alogarithmic curve around the ±3% of screen width, the amount of error inthe disparity around this area is low due to the roll-off of thelogarithmic function. This means that the amount of error introduced bycompanding in this region used for content being displayed on televisionor in a movie theatre is low. However, as the level of disparityincreases and approaches the ±1 frame width, the amount of errorincreases. But, as this level of disparity is used less frequently for3D content being displayed in a movie theatre or television, this erroris tolerable.

Clearly, the logarithmic encoding function can be selected depending onthe size of screen upon which the 3D content will be displayed. Forexample, given a smartphone has a small screen, it is unlikely that adisparity of ±3% would be used and that the disparity would be larger asa percentage of screen width. Therefore in this case, the logarithmicfunction can be selected that would increase the accuracy of thecompanding in the region used by the screen size upon which the 3D is tobe displayed.

So, in the example of companding a luminance value used to represent adisparity map for 3D content to be displayed on a TV or in a movietheatre, it is further possible to use the companding function

$\begin{matrix}{{F(x)} = {512 + {{round}\left( {512\mspace{14mu}{{sign}(x)}\frac{\ln\left( {1 + {\mu{\frac{x}{Dispscale}}}} \right)}{\ln\left( {1 + \mu} \right)}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu}(2)} \right\rbrack\end{matrix}$

Between the limits—Dispscale≦x≦+Dispscale

Where Dispscale is the number of disparity units; μ is a variable thatcontrols the magnitude of the logarithmic function. In embodiments, thevalue of μ is either 255 or 1023, although any value of μ may be usedwithin the limits of table 1 explained hereinafter. The term sign(x)means±(x) and the function round rounds the function to the nearestinteger value.

Accordingly, the decoding function is

$\begin{matrix}{{F^{- 1}(y)} = {{round}\left( {{Dispscale}\mspace{14mu}{{sign}\left( {y - 512} \right)}\left( \frac{1}{\mu} \right)\left( {\left( {1 + \mu} \right)^{\frac{y - 512}{512}} - 1} \right)} \right)}} & \left\lbrack {{Equation}\mspace{14mu}(3)} \right\rbrack\end{matrix}$

Where the variables have already been defined above with respect to theencoding function.

As noted above, the companding of the 17 bit disparity data into a 10bit luminance signal results in an error. This error is defined asError=x−F ⁻¹(F(x))  [Equation 4]

In other words, the error is calculated by firstly decoding thecompanded disparity map data and then calculating the difference betweenthe decoded (sometimes referred to as “expanded”) data and the originaldata. As noted above, in embodiments, the error will be smaller fordisparities that are more common for TV and movie theatre content thanfor larger, less often used values of disparity using the logarithmicfunction.

Although the companded 17 bit disparity value is reduced to 10 bitsusing equation (2) and sent over the luminance channel to the visionmixer 120, the disparity map will contain errors. This is due tocompanding being a lossy process. However, as companding is performed sothat the compression applied to the luminance value representingcommonly used disparity values, the error in the disparity maptransferred over the 10 bit luminance channel is reduced.

In order to remove the error completely, the error signal must be fed toa vision mixer according to embodiments of the present disclosure. Thefollowing will describe when the error signal is fed to the vision mixeraccording to embodiments. The error value is fed to the vision mixer 120of embodiments of the disclosure using the 10 bit chrominance channel.The error value will be used by the vision mixer 120 to recreate thecorrect value of luminance as will be explained with reference to FIG. 4which shows a vision mixer according to an embodiment of the presentdisclosure.

In order to ensure that the original signal can be re-created, the errorvalue must not exceed the 10 bits available on the chrominance channel.To ensure that the error value does not exceed the 10 bits available onthe chrominance channel, the value of μ and DispScale in equation 2 mustbe carefully selected. In particular, the maximum value of μ is selectedin dependence upon the value of DispScale. Table 1 shows therelationship between the maximum value of μ and the value of DispScaleto ensure that the error value does not exceed 10 bits available on thechrominance channel.

TABLE 1 DispScale (number of Disparity Bits) Maximum μ 30720 (16 Bits) ∞61440 (17 Bits) 4997 122880 (18 Bits)  64

Referring to FIG. 3B, an embodiment of the HD-SDI convertor 315 will bedescribed. The HD-SDI convertor 315 contains a 17 bit first look-uptable (LUT) 315A and a 17 bit second LUT 315B. The 17 bit disparity mapis fed into both the first LUT 315A and the second LUT 315B. The firstLUT 315A contains for each value of the disparity map (x) thecorresponding value (F(x)) defined by equation 2. Therefore, for anyvalue 17 bit value of disparity, the appropriate 10 bit luminance signalis output over the luminance channel. Similarly, the 17 bit luminancevalue is fed into the second LUT 315B. The second LUT 315B stores, foreach value of the disparity map (x) the corresponding error valuedefined by equation 4. The use of look-up tables allows operation sothat the value of F(x) and Error does not need to be calculated on thefly, and is quickly obtained.

Referring to FIG. 4, the 10 bit luminance signal and the 10 bit errorsignal (fed in to the vision mixer 120 over the 10 bit chrominancechannel) are fed into a disparity map decoder 405. The disparity mapdecoder 405 applies the decoding function of equation 3 to the 10 bitluminance signal to expand the companded luminance value. Similarly tothe HD-SDI convertor 315, the disparity map decoder 405, in embodiments,comprises a 10 bit look-up table which has the values of F⁻¹(y) storedin association with a value of the received 10 bit luminance value.

As noted hereinbefore, the value of F⁻¹(y) will contain errors. In orderto correct the error for each value output from the 10 bit look-uptable, the error value fed to the vision mixer using the chrominancechannel will be added by the decoder 405 to the value of F⁻¹(y) outputfrom the 10 bit look-up table. As the 10^(th) bit of the F⁻¹(y) is asign bit, no further processing of the error value is required and asimple addition of the error value and F⁻¹(y) need be performed.

In the vision mixer 120 of embodiments, the disparity map decoder 405 isconnected to a vision mixer controller 410. Typically, the vision mixer120 will have multiple video streams, captions and disparity maps inputthereto. The vision mixer controller 410 is controlled using softwarestored on vision mixer storage medium 425. The vision mixer controller410 receives the disparity map from each input source and providescontrol signals to a caption location device 415. The caption locationdevice 415 is configured to determine the location in video content toinsert the graphic or caption on the basis of the output of thedisparity map decoder 405. The caption location device 415 alsodetermines the depth at which the caption should be inserted. This isachieved by selecting a minimum disparity, from the luminance data, andproviding instructions to a 3D caption creation system (not shown) toplace the caption at the selected location and at the selected depth.

As will be appreciated by the skilled person, the disparity map decoder405 may be integrated into a vision mixer device according toembodiments or may be a separate device connected to a vision mixer.

Referring to FIG. 5, a flow diagram explaining embodiments of thepresent disclosure is disclosed. Typically the flow diagram will befollowed by a computer program according to embodiments of the presentdisclosure. In step 501 the disparity map is generated by controller 310(although the disparity map may be provided externally to the system).Each pixel of the disparity map in one or both of the left and right eyeimages is converted into a 17 bit luminance value. This is step S502.The 17 bit value is converted into a 10 bit luminance value and an errorvalue in step S503.

The 10 bit luminance value is fed to a vision mixer in step S504. Adecision is made at step s505. If 10 bit resolution at the vision mixeris sufficient the “yes” path is followed and the disparity map is usedto locate the position of the graphic or caption in step S506. If thevision mixer requires 17 bit resolution of the disparity map, theluminance representation is generated and the error value is applied tothe representation to correct for the error. This occurs in step S507.The corrected disparity map is then used in vision mixer in step S506 todetermine the location of the caption or graphic in video content.

It should be noted that although the above describes the compandingfunction as a logarithmic function, in embodiments the compandingfunction may be a segmented linear function instead. In this case,instead of a logarithmic function being used to reduce the 17 bitluminance value into the 10 bit companded luminance value, the fullrange of DispScale (i.e. ±61440 disparity units) is divided into aplurality of segments. In this embodiment and for ease of explanation,the segmented linear function has two segments. The gradients of thefirst segment and the second segment, in embodiments, are different. Inparticular, the gradient of the second segment is higher than thegradient of the first segment. The second segment is used to compand theluminance values associated with high levels of disparity and the firstsegment is used to compand the luminance values associated with lowerlevels of disparity.

For example, as noted above with reference to the logarithmic function,the lower levels of disparity are used in production of 3D content fordisplay on TVs or in a movie theatre. More specifically, the firstsegment is used to compand approximately ±3.3% of the screen width andthe second segment is used to compand between ±3.3% of the screen widthand full screen width. As the gradient of the second segment is higherthan the gradient of the first segment, the error associated withcompanding using the second segment is higher than the error associatedwith companding using the first segment. This means that the amount oferror associated with luminance values representing a high disparitybetween the left image and the right image is higher for a given sizecompanded luminance value.

In embodiments, the second segment has a gain gradient of 256 (or 2⁸)and the first segment has a gain gradient of 8 (or 2³). Accordingly,first segment loses the lowest 3 significant bits during companding andthe second segment loses the lowest 8 significant bits duringcompanding. Clearly, although the gain gradients have been specified as8 and 256 respectively, any value may be selected. However, similarly tothe logarithmic example, it is possible to select a smaller gradient forthe most commonly used disparity to reduce the amount of error.

In embodiments, the companding function for the first segment is

$\begin{matrix}{{F(x)} = {{512 + {{round}\left( \frac{x}{8} \right)}\mspace{56mu} - 2048} < x < 2048}} & \left\lbrack {{equation}\mspace{14mu} 5} \right\rbrack\end{matrix}$

And the companding function for the second segment is

                                      [equation  6] $\begin{matrix}{{F(x)} = {512 + {{round}\left( {\frac{x}{256} + {248\mspace{14mu}{{sign}(x)}}} \right)}}} & {{- {DispScale}} \leq x \leq {- 2048}} \\\; & {2048 \leq x \leq {DispScale}}\end{matrix}$

Where like variables are already defined in the logarithmic function.

The error function is the same as equation 4 above.

The decoding function associated with equation 5 isF ⁻¹(y)=8(y−512)256<y<768  [equation 7]

And the decoding function associated with equation 6 is

$\begin{matrix}\begin{matrix}{{F^{- 1}(y)} = {256\left( {y - 512 - {248\mspace{14mu}{{sign}\left( {y - 512} \right)}}} \right)}} & {24 \leq y \leq 256} \\\; & {768 \leq y \leq 1000}\end{matrix} & \left\lbrack {{equation}\mspace{14mu} 8} \right\rbrack\end{matrix}$

In the case of a decoder being required to expand the compandedfunction, it may be necessary for the encoder and decoder to exchangeinformation indicating which encoding function is used to compand thedisparity value. This exchange may, for example, occur during ahandshaking protocol when the decoder and encoder are first connected ormay be exchanged by adding an Ancillary data packet to the HD-SDI streamduring encoding to indicate to the decoder which encoding scheme isused. Alternatively, this information may be provided by the user or thelike.

However, it should be noted that the decoder and encoder may only needto exchange this information when different companding techniques areused. For example, if a decoder and encoder are configured to use boththe logarithmic technique and the segmented linear technique. Further,in the event that 10-bit resolution provides sufficient accuracy for thevision mixer, no exchange is required.

Although the foregoing includes a disparity map being generated in thedevice, the disclosure is no way limited to this. Indeed, the creationof the disparity map itself should not limit the disclosure. Forexample, the disparity map may be created by a CGI animation package, ormay have been previously captured and stored on a disk or memory deviceor the like.

Although illustrative embodiments of the disclosure have been describedin detail herein with reference to the accompanying drawings, it is tobe understood that the disclosure is not limited to those preciseembodiments, and that various changes and modifications can be effectedtherein by one skilled in the art without departing from the scope andspirit of the disclosure as defined by the appended claims.

In so far as embodiments of the disclosure have been described as beingimplemented, at least in part, by software-controlled data processingapparatus, it will be appreciated that a non-transitory machine-readablemedium carrying such software, such as an optical disk, a magnetic diskor the like is also considered to represent an embodiment of the presentdisclosure.

It will be appreciated that the above description for clarity hasdescribed embodiments with reference to different functional units,circuitry and/or processors. However, it will be apparent that anysuitable distribution of functionality between different functionalunits, circuitry and/or processors may be used without detracting fromthe embodiments.

Described embodiments may be implemented in any suitable form includinghardware, software, firmware or any combination of these. Describedembodiments may optionally be implemented at least partly as computersoftware running on one or more data processors and/or digital signalprocessors. The elements and components of any embodiment may bephysically, functionally and logically implemented in any suitable way.Indeed the functionality may be implemented in a single unit, in aplurality of units or as part of other functional units. As such, theinvention may be implemented in a single unit or may be physically andfunctionally distributed between different units, circuitry and/orprocessors.

Although the present disclosure has been described in connection withsome embodiments, it is not intended to be limited to the specific formset forth herein. Additionally, although a feature may appear to bedescribed in connection with particular embodiments, one skilled in theart would recognize that various features of the described embodimentsmay be combined in any manner suitable to implement the invention.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims priority to United Kingdom ApplicationGB1205758 filed on 30 Mar. 2012, the contents of which beingincorporated herein by reference in its entirety.

The invention claimed is:
 1. A method comprising: obtaining, bycircuitry, a disparity map for one of a left eye image and a right eyeimage, the disparity map indicating the horizontal displacement betweencorresponding pixels in the left eye image and the right eye image;generating, by the circuitry, a luminance representation of thedisparity map, wherein a disparity unit which defines the resolution ofthe disparity map, is mapped to a corresponding luminance value;companding, by the circuitry, the luminance value for transmission overa luminance channel; generating, by the circuitry, an error signal, theerror signal being the difference between the companded luminance valueand the luminance value; and converting, by the circuitry, the errorsignal for transmission over a chrominance channel.
 2. The methodaccording to claim 1, wherein the luminance is companded using alogarithmic function.
 3. The method according to claim 1, wherein theluminance is companded using a segmented linear function.
 4. The methodaccording to claim 2, wherein the companding function is selectedaccording to the size of the display onto which the left eye image andthe right eye image are to be stereoscopically displayed so that thegradient of the companding function is greatest at a disparity suited tothe display${F(x)} = {{512 + {{{round}\left( {512\mspace{11mu}{sign}\mspace{11mu}(x)\frac{\ln\left( {1 + {\mu{\frac{x}{Dispscale}}}} \right)}{\ln\left( {1 + \mu} \right)}} \right)}{F(x)}}} = {{512 + {{{round}\left( \frac{x}{8} \right)}{F(x)}}} = {512 + {{{round}\left( {\frac{x}{256} + {248\mspace{14mu}{{sign}(x)}}} \right)}.}}}}$5. The method according to claim 1, further comprising expanding thecompanded luminance value; retrieving the error signal from thechrominance channel and adding the retrieved error signal to theexpanded luminance value to form the original luminance value.
 6. Themethod according to claim 1, comprising retrieving the compandedluminance value from a look-up table where the companded luminance valueis stored in association with the luminance value.
 7. The methodaccording to claim 1, comprising retrieving the error signal from a lookup table where the error signal is stored in association with theluminance value.
 8. A non-transitory computer-readable medium includingcomputer program instructions, which when executed by a device, causesthe device to perform a method according to claim
 1. 9. An encodercomprising: disparity map obtaining circuitry configured to obtain adisparity map for one of a left eye image and a right eye image, thedisparity map indicating the horizontal displacement betweencorresponding pixels in the left eye image and the right eye image; aluminance value generator circuitry configured to generate a luminancerepresentation of the disparity map wherein a disparity unit, whichdefines the resolution of the disparity map, is mapped to acorresponding luminance value; a companding circuitry configured tocompand the luminance value for a luminance channel; an error signalgenerator circuitry configured to generate an error signal, the errorsignal being the difference between the companded luminance value andthe luminance value; and an error signal converting device circuitryconfigured to convert the error signal for transmission over achrominance channel.
 10. The encoder according to claim 9, wherein theluminance is companded using a logarithmic function.
 11. The encoderaccording to claim 9, wherein the luminance is companded using asegmented linear function.
 12. The encoder according to claim 10,wherein the companding function is selected by the companding circuitryaccording to the size of the display onto which the left eye image andthe right eye image are to be stereoscopically displayed so that thegradient of the companding function is greatest at a disparity suited tothe display${F(x)} = {{512 + {{{round}\left( {512\mspace{11mu}{sign}\mspace{11mu}(x)\frac{\ln\left( {1 + {\mu{\frac{x}{Dispscale}}}} \right)}{\ln\left( {1 + \mu} \right)}} \right)}{F(x)}}} = {{512 + {{{round}\left( \frac{x}{8} \right)}{F(x)}}} = {512 + {{{round}\left( {\frac{x}{256} + {248\mspace{14mu}{{sign}(x)}}} \right)}.}}}}$13. The encoder according to claim 9, comprising a look up table,wherein the look up table is configured in use to store the compandedluminance value in association with the luminance value, and thecompanding device circuitry is configured in use to retrieve thecompanded luminance value from the look-up table.
 14. The encoderaccording to claim 13, wherein the look-up table is configured in use tostore the error signal, where the error signal is stored in associationwith the luminance value and the companding device circuitry is furtherconfigured to retrieve the error signal from the look-up table.